Skip to content

perf(api): build related_fields' nested serializers once per page - #16992

Merged
blakeaowens merged 2 commits into
devfrom
perf/finding-nested-serializer-reuse
Nov 18, 2026
Merged

blakeaowens merged 2 commits into
devfrom
perf/finding-nested-serializer-reuse

Conversation

@blakeaowens

@blakeaowens blakeaowens commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Description

A 1000-row GET /api/v2/findings/?related_fields=true page spent most of its wall time in Python, and with JIRA enabled it also ran two queries per finding that has a JIRA issue.

  • FindingSerializer.get_related_fields built a new FindingRelatedFieldsSerializer for every row, and that built a new FindingTestSerializer (three levels of nested serializers) and a new JIRAIssueSerializer for every row too. Building a serializer builds its whole field tree, and nothing in those trees depends on the row. Each is now built once per serializer instance (a cached_property) and reused for every row the list renders, the same pattern perf(api): load the relations of prefetched duplicate findings once per page #16789 used in the prefetcher. A page without related_fields=true still builds none of them.
  • related_fields.jira.url reads the issue's JIRA project and its instance. Nothing loaded them with the page, so each issue cost two queries. FindingViewSet.get_queryset now prefetches jira_issue__jira_project__jira_instance, only when related_fields=true, so plain pages run exactly the statements they ran before. An issue with no project of its own (older records) renders its url from the engagement's project, else the product's, so those are prefetched too; and nothing JIRA is prefetched while JIRA is disabled, where the url is never read.

Response bodies are byte-identical before and after.

Measured on 1000 findings (a third with a JIRA issue linked to a project), median of 9 requests, before and after under the same load:

1000-row page before after
superuser, related_fields=true 9,090 ms, 700 statements 4,854 ms, 34 statements
product Owner, related_fields=true 8,894 ms, 706 statements 5,303 ms, 40 statements
superuser, plain 4,179 ms, 32 statements 4,257 ms, 32 statements

A profile of the before request put about 6.3 s (under the profiler) in DRF get_fields, 6,537 calls, almost all from get_test.

Test results

New unittests/test_apiv2_finding_related_fields_reuse.py:

  • each of the three nested serializers is built once for a 2-row page and for a 6-row page (on dev: twice for 2 rows),
  • related_fields adds the same statements to 2 rows as to 6, measured against the plain page (on dev: 4 and 12),
  • the payload is byte-identical to the per-row construction, with and without a JIRA issue,
  • the JIRA prefetches (the issue's own project and the engagement and product fallbacks) are added for related_fields=true while JIRA is enabled, and not for a plain page or while JIRA is disabled; odd rows in the statement test have issues with no project of their own.

2 of the 5 fail on dev for those reasons, and all 5 pass with the change. The paired downstream suite (finding list query scaling and performance tests) passes unchanged.

Documentation

No documentation change: nothing a user sees or configures changes.

Checklist

  • Make sure to rebase your PR against the very latest dev.
  • Submit all PRs, features and bug fixes alike, against the dev branch.
  • Give a meaningful name to your PR, as it may end up being used in the release notes.
  • Your code is Ruff compliant (see ruff.toml).
  • Your code is python 3.13 compliant.
  • If this is a new feature and not a bug fix, you've included the proper documentation in the docs at https://github.com/DefectDojo/django-DefectDojo/tree/dev/docs as part of this PR.
  • Model changes must include the necessary migrations in the dojo/db_migrations folder.
  • Add applicable tests to the unit tests.
  • Add the proper label to categorize your PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HJqD5EJ6LWVNAPF4qew47G

A 1000-row ?related_fields=true findings page spent most of its time
building serializers: get_related_fields built a FindingRelatedFieldsSerializer
for every row, which built a FindingTestSerializer (three nested levels)
and a JIRAIssueSerializer for every row too. Each is now built once per
serializer instance and reused for every row.

related_fields' jira url read each issue's JIRA project and instance with
a query each, so a related_fields page now prefetches
jira_issue__jira_project__jira_instance. Plain pages are unchanged.

Response bodies are byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HJqD5EJ6LWVNAPF4qew47G
@blakeaowens blakeaowens added this to the 3.5.300 milestone Nov 18, 2026
… enabled

A JIRA issue with no project of its own (older records) renders its url
from the engagement's project, else the product's: those are now
prefetched with a related_fields page too, so they cost no query per
row either. Nothing JIRA is prefetched while JIRA is disabled, where
the url is never read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HJqD5EJ6LWVNAPF4qew47G
@blakeaowens
blakeaowens added this pull request to the merge queue Nov 18, 2026
Merged via the queue into dev with commit e243677 Nov 18, 2026
49 checks passed
@blakeaowens
blakeaowens deleted the perf/finding-nested-serializer-reuse branch November 18, 2026 23:12
@defectdojo-release-tracker defectdojo-release-tracker Bot modified the milestones: 3.5.300, 3.5.201 Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants